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Introduction 

Under Paragraph 223 of the Consent Decree, the Court retains jurisdiction over this 
matter "until such time as the City has achieved full and effective compliance and 
maintained such compliance for no less than two years." On January 10, 2018, the Court 
entered an order finding the Department to be in "full and effective compliance" as of 
the date of the Order, thus commencing at that point the two-year "sustainment period." 
Dkt. #439. The Court further ordered the parties and the monitor to "meet, confer, and 
prepare a plan for discharging their obligations under the Consent Decree" during this 
two-year period. 

On March 13, 2018, the Court entered an order approving the Sustainment Period Plan 
developed pursuant to the Court's January 10 th order. This plan, and an attached matrix 
of deadlines, became the governing documents for this Sustainment Period. 

As relates to SPD's obligations under the Consent Decree, the matrix contains deadlines 
for SPD self-reporting or assessment of specific topical areas ("Commitments") of the 
Consent Decree. These assessments, implicated paragraphs of the Consent Decree, and 
associated deadlines are provided in the chart that follows: 


Sustainment 

Matrix 

Deliverable 

Nos. 

Systemic 

Report/Audit/Assessment 

Filing Deadline 

6 

Community Engagement 

May 31, 2018 

114 

Stops and Detentions - Outcome 

June 30, 2018 

120 

Report 

May 31, 2019 

134 

Stops and Detentions - Audit 

January 31, 2019 

142 

Report 

October 31, 2019 

99 

Crisis Intervention - Outcome 

October 31, 2018 

111 

Report 

October 31, 2019 

172 

Supervision 

October 31, 2018 

180 


November 30, 2019 

25 

Force Reporting, Review, and 

October 31, 2018 

33 

Investigation - Type 1 

October 31, 2019 

42 

Force Reporting, Review, and 

October 31, 2018 

50 

Investigation - Type II 

July 31, 2019 
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Sustainment 

Matrix 

Deliverable 

Nos. 

Systemic 

Report/Audit/Assessment 

Filing Deadline 

105 

Crisis Intervention/Use of Force 

December 15, 2018 

56 

Use of Force - Outcome Report 

January 31, 2019 

62 


January 10, 2020 

79 

Force Review Board 

July 31, 2019 

71 

Use of Force - Comprehensive 

October 31, 2019 

194 

Early Intervention 

March 15, 2019 

202 


December 1, 2019 

156 

Bias-Free Policing - Disparity 
Review (Use of Force and 
Search/Seizure) 

April 30, 2019 

164 

Bias Free Policing - Disparity 
Review (Use of Force/Search and 
Seizure/other activity) 

December 31, 2019 


This report covers SPD's obligations with respect to Sustainment Matrix Deliverable No. 
194. Following the agreed methodology attached as Appendix A to this report, this report 
demonstrates SPD's sustained compliance with those commitments of the Consent 
Decree (paragraphs 157-163) that relate to Early Intervention. 

I. EIS - Consent Decree Requirements 

The Consent Decree between the City of Seattle and the Department of Justice contains 
seven paragraphs relating to EIS, reproduced here: 

157. "The City's EIS system will continue to be used for risk management 
purposes and not for disciplinary purposes. SPD will monitor the EIS to 
ensure it is meeting its objective of providing SPD with notice before 
behaviors become problematic. 

158. SPD will review and adjust, where appropriate, the threshold levels for each 
of the current EIS indicator criteria, and the EIS indicators. The Monitor will 
review and approve the revised EIS threshold levels and indicators. 

159. SPD will revise its EIS policy to include a mechanism for review of an officer 
whose activity has already triggered a threshold for one of the EIS indicator 
criteria, so that the threshold level is lower if EIS is triggered again, where 
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appropriate. For example, if an officer has participated in a certain number 
of uses of force in a six-month period, SPD will design a protocol for lowering 
the threshold for subsequent review. 

160. SPD will collect and maintain information related to supervisor, precinct, 
squad and unit trends, consistent with the provisions of this section. 

161. SPD will collect, maintain, and retrieve information related to the following 
precinct-level activity: 

a) uses of force; 

b) OPA complaints and their dispositions; 

c) number of individual officers who have triggered EIS reviews; and 

d) supervisor reviews with officers. 

162. Supervisors should periodically review EIS activity of officers in their chain 
of command. 

163. SPD will revise its EIS policy and procedure, as necessary, so that 
interventions assist officers in avoiding potentially troubling behavior. 
Specifically, SPD policies and procedures will ensure that (1) the 
intervention strategy is implemented in a timely manner; (2) data regarding 
the implementation of the intervention is tracked in EIS; and (3) if 
necessary, the employee's supervisor reviews the progress of the 
intervention strategy." 

Consent Decree HU 157-163. 

SPD's compliance with these provisions is discussed below. 


II. SPD’s EIS Policy 

To satisfy these requirements of the Consent Decree, SPD implemented a new EIS 
policy in mid 2015. See Monitor's Third Semiannual Report (dkt. 123) at 76-89. The 
policy was approved by the U.S. Department of Justice, the Monitor and the Court. Id. 

SPD's EIS policy relies on triggers, thresholds, and alerts. The following definitions are 
instructive. 
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Trigger: This is the basic building block of EIS. The system monitors the 
occurrence and frequency of triggers for each officer. Triggers are either 
individual actions by an officer (e.g., an officer uses force) or an event related to 
an officer's actions (e.g., an OPA complaint or lawsuit is filed against an officer). 
SPD's EIS policy refers to these actions or events as "triggers," "indicators," and 
"indicator criteria." 

Alert: Alerts occur when an employee reaches a predetermined number of 
triggers within a set time period. The EIS module of lAPro (one of SPD's case 
management systems) generates the alert, which is received electronically by the 
EIS Coordinator. The Alert contains a listing of the triggers that caused the Alert 
and details about the employee, such as name and assignment. 

Threshold: SPD's EIS is threshold based. The threshold is the number of triggers 
and the time period that cause an Alert. For example, the threshold for 
preventable vehicle collisions is 2 collisions within 6 months. 

Early Intervention Assessment: The EIS Coordinator receives an Alert and then 
assigns an Assessment that must be completed by the chain of command. The 
Assessment document makes a recommendation as to whether the officer 
should be placed on a mentoring plan. As explained in Section III.C, not all Alerts 
lead to an Assessment. 

Early Intervention Mentoring Plan: When a chain of command determines that 
the reviewed triggers represent a need for performance improvement. A 
mentoring plan will be created when, in the judgement of the chain, the 
performance could be mitigated with a defined curriculum. The plan will be 
written by the employee's direct supervisor and receive command approval to 
the deputy chief level. In addition, the chain of command can initiate a 
mentoring plan at any time based on their own observed concerns, even if an 
employee has not alerted. 

The Performance Review Committee (PRC): The PRC includes both members of 
Command Staff and Department representatives across multiple bureaus, 
including the Chief's designee (Deputy Chief), the Executive Director of Human 
Resources, the Executive Director of Legal Affairs, the Early Intervention 
Coordinator, and representatives from Patrol Operations, the Audit, Policy and 
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Research Section, and the Education and Training Section. The PRC meets 
monthly and, per policy and by practice, is responsible for: 

• Ensuring Department-wide consistency and uniformity in the 
implementation of the EIS, including timeliness of Assessments 
consistent with policy requirements; 

• Determining whether Assessments and proposed Early Intervention 
Mentoring Plans are adequate to address concerns about risk 
factors, demonstrated indicators of stress, or training opportunities, 
and to provide the employee with any necessary tools and support. 

EIS Coordinator : a supervisor-level position housed in Human Resources, 
responsible for tracking and routing EIS Alerts, monitoring timelines associated 
with EIS review deadlines, and tracking the dispositions of EIS Assessments 
(also available in DAP). 

The Consent Decree requires SPD to monitor, review, and adjust, where appropriate, 
the EIS triggers and thresholds. HU 157-58, 163. On January 1, 2019, the triggers and 
thresholds were revised, with approval from the U.S. Department of Justice, the 
Monitor, and the Court. Dkts. 502 & 510. The rationale and goals for these changes, 
and other policy changes intended to improve the EIS program, are explained at length 
below. 


During the Study Period—January 1, 2017 to June 30, 2018—the triggers and 
thresholds were as follows: 


Trigger 

Threshold Levels 

A. Chain-of-command recommendations / 
discretionary assessments 

Each will be reviewed 

B. Use-of-force (Type 1) 

Reaching the top 1% of officers who have used 
force investigated at Type 1 within 6 months 

C. Use of-force (Type II) 

Reaching the top 5% of officers who have used 
force investigated at Type II within 6 months 

D. Use-of-force (Type III) 

Reaching the top 5% of officers who have used 
force investigated at Type III within 6 months 

E. Vehicle collisions 

2 Department vehicle collisions within 12 
months 
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F. Receipt of OPA complaints / Supervisory 

Action 

3 complaints within 12 months 

G. Receipt of OPA complaints / Frontline 
Investigation 

3 complaints within 12 months 

FI. Receipt of OPA complaints / OPA 

Investigation 

3 complaints within 12 months 

1. Receipt of EEO complaints 

2 complaints within 12 months 

J. Named in police actions claims or Police action 
lawsuit against the City 

2 within 24 months 

K. Vehicle pursuits 

2 within 6 months 

L. Unexcused failure to appear in mandatory 
training 

1 within 12 months 

M. K9 apprehension-bite ratio 

More than 15% K-9 apprehension bite ratio in a 

12 month period 

N. Officer-involved shooting 

Single incident threshold 


Additionally, an Early Intervention Assessment will be conducted at the aggregate level 
when an employee has a total of six (6) indicators between A - M during any six-month 
window when the employee is not on a mentoring plan. 


III. Objectives and Methodology 

This report covers a study period of January 1, 2017, to June 30, 2018. Data for this report 
are derived from three sources. Data on supervisor reviews and individuals who have 
triggered EIS reviews is contained in records maintained by the EIS Coordinator. Data 
regarding precinct and squad trends with respect to use of force and closed OPA 
complaints/dispositions are sourced from the Data Analytics Platform (DAP, as described 
in detail in prior reports). 
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Data is also drawn from lAPro, the Department's database solution for records and case 
management relating to activities that may result in EIS triggers. lAPro provides the 
technical administration of EIS (e.g., it generates alerts and tracks workflow); the DAP 
collects EIS alert and workflow information from lAPro and renders it for analysis. While 
some additional data may be joined to provide additional analytical context for the EIS 
"incident," 1 the record of the alert is discrete and generated programmatically, resulting 
in a high-quality / high-confidence record that is not subject to functional error handling 
rules. 2 Of note, when the EIS code was delivered to the Department by Accenture, the 
contracted vendor of the DAP in Fall 2016, the code was accepted only after passing exit 
criteria 3 for technical testing and User Acceptance Testing (UAT). 

A. Objective 1 - Assess Whether EIS Is Accomplishing Its Goals (Paragraphs 
157 - 159 ; 163 ) 

SPD is committed to continuously evaluating the effectiveness of EIS and making 
evidence-based improvements as warranted. 

The EIS program is successful at developing supervisory skills and providing effective 
coaching and mentoring to officers. Each Early Intervention Assessment that is completed 
requires the officer and his or her supervisor to meet in person to discuss the incidents 
that caused the alert. These assessments, alone, document a large volume of active, 
engaged supervision that otherwise would not have occurred. Mentoring plans also 
require supervisors to problem solve and engage with their employees on an ongoing 
basis, often over a period of several months. Another feature of the program is that the 
EIS Coordinator serves as an important resource to supervisors. In addition to reviewing 
and providing feedback on supervisors' written assessments and mentoring plans, the EIS 


1 lAPro captures threshold exceedances through the creation of new "incidents" and adds every new EIS 
alert to the incidents table in the lAPro database. 

2 The DAP maintains rules for functional error handling intended to protect the analysis from "orphan" or 
partial records. For data concept areas such as UoF, where multiple complete elements are required for 
a valid count (i.e. unique incident, officer and community member), a functional error may be applied, 
flagging the record for correction. With regard to EIS, data convergence occurs entirely within the lApro 
system (a trigger cannot occur without a complete record of a UoF, for example), functional error handling 
is therefore not applicable. 

3 Technical testing involved source-to-target testing at every layer of the DAP system. UAT acceptance 
criteria required the delivered product (code and semantic layer) function analytically, sufficient to 
respond to questions necessary to demonstrate compliance. 
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Coordinator also maintains a repository of reference materials including training videos, 
books, and sample mentoring plans. 

Despite these successes, SPD has identified shortcomings in the EIS program. In an 
extensive effort to address these issues, over the past several months, SPD implemented 
numerous, substantial policy changes which were approved by DOJ, the Monitoring 
Team, and the Court. 

1. Has SPD monitored EIS to determine whether EIS is meeting its risk 
management objectives? (Paragraphs 157 and 163) 

During the study period, in order to evaluate the effectiveness of SPD's EIS in achieving 
its risk management objectives, SPD partnered with a team of researchers at Washington 
State University to compare overall officer performance across a series of behaviors, 
based on interval-level metrics, between a group of officers who received an EIS alert and 
a group of officers who did not. A copy of the report, detailing the full methodology and 
results, is attached as Appendix B. Using a blinded scoring design (where study personnel 
assigned to scoring officer performance were not aware of which reports belonged to the 
treatment group and which belonged to the control group), researchers examined a set 
of 500 randomly selected general offense reports generated from incidents that resulted 
in EIS alerts between 2015-2017 and a set of 500 randomly selected reports from this 
same time period that did not result in an EIS alert. 

Researchers evaluated the incident reports and used a blind-scoring design to calculate a 
performance score for each officer. The performance scores were based on three sets of 
metrics: use of force, tactical social interaction, and crisis intervention. For example, a 
scorer would determine, if applicable to the situation, whether the officer actively 
listened to the person in crisis or whether the officer demonstrated proficiency with less- 
lethal force options. The performance scoring is explained at pages 4-12 of the report. 

As of the time of this filing, WSU's report has been submitted to an academic journal 
for publication; throughout this process it will be peer reviewed. 

The report concluded that the EIS triggers in place at the time did not adequately 
account for the individual officers' exposure rates to high-risk encounters and situations 
and, as a result, the triggers focused too much on officers who are more proactive and 
who are assigned to work in areas with a higher volume of calls for service. 

Additional salient findings include: 
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• No significant difference between ElS-flagged and control officer total 
performance score. 

• The only significant difference in performance scores was that EIS flagged officers 
had better performance scores on the "observe/assess" level during crisis 
encounters. The "observe / assess" category relates to officers' ability to accurately 
read people during interactions. 

• ElS-flagged officers were significantly more likely to report using (in policy) force 
than other officers (17.5% vs. 11.7%). 

The second unintended consequence of the EIS program that SPD identified is anxiety 
and negative perceptions on the part of officers. To both complement the empirical 
study with WSU and to inform work SPD is partnering in at the national level, SPD also 
sought feedback, in focus groups, from individuals directly engaged with the system - 
including members of the Performance Review Committee, patrol officers, and 
supervisors and commanders. The following themes were consistently expressed. 

• Officers and supervisors believe that EIS does not reliably identify officers at risk of 
future problematic performance. 

• Officers experience anxiety over the prospect of being more likely to receive a 
higher number of EIS alerts because of their patrol assignment. 

• Because EIS review is triggered by incidents that are already subject to intense 
review and investigation by the chain of command and/or by OPA ( e.g. use of force 
reports, OPA complaints), EIS is perceived by officers and supervisors alike as 
redundant. To be sure, it is important to keep the review processes that can lead 
to discipline separate from the EIS program, because the goals of the EIS program 
are best accomplished through coaching, mentoring, and support—not discipline. 
Nonetheless, supervisors and officers experience the dual review as repetitive. 

• EIS review of Type I uses of force, in particular, is seen as an unnecessary drain on 
supervisory resources. The data show the reason for this perception. During the 
study period, Type I force accounted for just shy of 40% of all alerts. Yet, in 2017, 
there were no sustained allegations of out of policy use of Type I force. 
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To address the issues identified by the WSU study, SPD implemented substantial 
changes to its policies, as described in section 2 below. 

2. Has SPD reviewed and adjusted EIS indicator criteria and thresholds as 
appropriate? (Paragraphs 157-158 and 163) 

Throughout the second half of 2018, SPD undertook a major effort to reform the EIS 
program and related policies. This work was undertaken in consultation with the 
Monitor and DOJ, and based on evidence and feedback from numerous sources. The 
policy revisions are informed by all of the following: (1) a review of EIS data over the 
past two years, both internally and as provided in the WSU study; (2) feedback from 
the Performance Review Committee; (3) focus group sessions with officer, sergeants, 
lieutenants, and captains; and (4) new analytic tools for aggregating and analyzing 
force at increasingly granular and comparative levels. 

The first set of policy changes involved revising the definition of Type I Force to clarify that 
discomfort caused by wearing (properly applied) handcuffs is not reportable as force. SPD 
had determined that such reports of discomfort or minor pain— i.e., the level of discomfort 
that is inherent to wearing handcuffs—made up an increasingly high percentage of 
reported use-of-force incidents (approximately 44% of Type I force in 2017 and two-thirds 
of Type I force in 2018). 4 The policy now distinguishes between pain resulting from the 
use of force involving handcuffs (e.g., improper application of handcuffs) versus pain that 
can be attributed solely to the inherent discomfort of wearing handcuffs. SPD's revised 
Use of Force policy defines "handcuff discomfort" as "discomfort or minor pain caused 
solely by wearing handcuffs properly applied." 

Accompanying that change is a revision which provides that such complaints must be 
reported and screened by a supervisor. These revisions are intended to ensure that all 
complaints of pain from handcuffs will be investigated as either a Type I use of force 
(application pain) or as Handcuff Discomfort (wearing handcuffs) and properly tracked by 
the Department. These changes were discussed at length with DOJ and the Monitor and 
approved by the Court. Dkt. 477. 

In addition, the Department recently tested and deployed new handcuffs which are 
beveled and made of lighter-weight aluminum, and which it anticipates will be equally 
effective and cause less discomfort. These policy and equipment changes are expected to 
reduce the amount of time and effort spent by the chain of command on instances where 
handcuffs, properly applied, cause unavoidable discomfort. 


4 See SPD's 2017 and 2018 Use of Force Outcome Reports, previously filed with the 
Court. Dkts 443 at 47 and 524-1 at 5. 
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Second, SPD worked, consistent with Paragraphs 158 and 163 of the Consent Decree, 
to adjust and revise EIS thresholds and indicators. As with the use of force policy 
revisions, SPD engaged in multiple rounds of discussions with the Monitor and the 
Department of Justice in developing the proposed revisions, and—with their 
acceptance—received approval from the Court on December 17, 2018 (Dkt. No. 510). 

SPD continues to be in compliance with this requirement. 

The revised EIS policy modified the description of the program's purpose, triggers, and 
risks to better align the policy with the philosophy of the program which is not intended 
to be punitive or disciplinary. For example, instead of describing the EIS program solely as 
a "strategy to address at-risk behavior," the revised policy identifies a more holistic set of 
goals, including to support employee wellness and professional growth and to address 
underlying factors that can lead to negative performance issues (including incident-based 
stress or training needs). The revised policy also clarifies that the purpose of a mentoring 
plan under EIS is to support the employee in achieving professional goals. 

In addition, the EIS policy revisions adjusted the trigger levels. To respond to the analysis 
and conclusions summarized above, the triggers for use of force are adjusted so that they 
are still based on percentages, but now are also subject to a baseline number of uses of 
force at or beneath which there will be no alert. For Type I force, the trigger is still the top 
1% of officers, with a baseline of five incidents. Similarly, for Type II the percentage stays 
at 5%, with a baseline of three incidents. The threshold for triggering review is raised from 
3 OPA complaints to 4 OPA complaints. The adjustments are intended to exclude from 
review the officers who fall within the top percentages for use of force, but still have used 
force very rarely (to account for the fact that Department-wide the rate of force being 
used is extremely low). 

Four of the trigger categories were eliminated, because they are duplicative of other policy 
provisions or review processes: (1) the unexcused failure to appear at training category 
was eliminated, because supervisors are already required elsewhere in policy to discuss 
any and all missed trainings with officers; (2) the K9 apprehension bite category was 
eliminated, because these incidents already trigger review under the Type II use of force 
category; (3) the vehicle pursuit category was eliminated, because it is already extensively 
reviewed by the chain of command pursuant to SPD's vehicle eluding/pursuits policy; and 
(4) similarly, the officer involved shooting (OIS) category was eliminated as an EIS trigger 5 
because the Department has adopted a far more extensive process for OIS incidents— 
after every OIS incident the Department places the involved officer on mandatory leave, 


5 Type III force is still a trigger, and under the current threshold any officer who exceeds 
one use of Type III force (including more than one OIS) within six months will be subject 
to an EIS alert. 
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the specially trained Force Investigation Team conducts an investigation, and an in-depth 
inquiry is undertaken by the Force Review Board. 

The Department expects that these changes will more closely focus the EIS program on 
activities that are relevant to risk and achieve better alignment with the program's goals. 

3. Does the EIS policy contain a mechanism by which the threshold for 
review is lower after an officer has already alerted? (Paragraphs 159 
and 163) 


SPD continues to be in compliance with paragraphs 159 and 163. The EIS policy as 
approved by the Court on December 17, 2018, retains the language of the Department's 
prior (Court approved) policy with respect to this requirement and states as follows: 

2. When an Officer or Civilian Employee Under an Early 
Intervention Mentoring Plan Has Additional Early Intervention 
Threshold Indicator Activity, That Employee's Lieutenant or 
Manager Will Review Any Further Early Intervention Threshold 
Indicator Activity 

If an officer currently the subject of an Early Intervention 
Mentoring Plan has additional Early Intervention System 
activity, the employee's lieutenant will review the new event 
and submit to the captain, via the status report, a written 
recommendation regarding whether the original Early 
Intervention Mentoring Plan should be revised. Additional 
entries to the Early Intervention Mentoring Plan due to 
additional triggers are reviewed by the chain-of-command and 
the Performance Review Committee. 


This policy is implemented by the EIS Coordinator. By design, once an officer has alerted 
(and therefore is already being assessed or on a mentoring plan), then the EIS Coordinator 
receives electronic notification for every new trigger that occurs for that officer. 6 (As 
noted above, policy uses the word "trigger" to refer to the occurrence of an incident or 
action that is tracked by EIS, e.g., a use of force or the filing of an OPA complaint.) The EIS 
Coordinator reads each new alert and reviews the employees' record to ensure that the 
alert is not repeating previous triggers (e.g., one new of pointing a firearm, when the 
officer is already being assessed or on a mentoring plan based on similar Type I uses of 
force). If the new alert is based on three or more new triggers, the officer's supervisor is 
required to complete an updated assessment (using a form titled Subsequent Alert 


6 This occurs by design, because the EIS alert system does not reset or "zero out" after an officer alerts 
until the officer is outside the measuring period (e.g., if an officer alerts for six Type I uses of force within 
six months, then each new Type I force will cause a new alert until the system "resets" six months after 
the first incident). The purpose is to ensure close and continuous monitoring of the officer's activities. 
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Update). However, if an officer who is already on a mentoring plan receives even one new 
trigger, then the officer's lieutenant or manager is required to perform an additional 
review. 

The purpose of an update assessment is to inform a supervisor that an employee the 
supervisor has recently completed an Assessment on has at least three new un¬ 
reviewed triggers. The supervisor is required to review the new triggers and assess 
whether or not the new information alters his or her assessment of the employee's 
performance when analyzed in conjunction with the previous history. 

To illustrate, assume an officer receives an "aggregate" alert based on six triggers: e.g., 
two instances of pointing a firearm, two complaints of handcuffing discomfort, and two 
OPA complaints. 7 An Assessment is assigned to the officer's supervisor. Within two 
months of the alert, the officer has three additional Type I uses of force. The EIS 
Coordinator would assign the supervisor to complete an update assessment (if the 
officer had not already alerted, it would take six Type I uses of force to cause an 
assessment). 

SPD accordingly continues to be in compliance with this requirement of the Consent 
Decree. 

B. Objective 2 - Assess Whether SPD Is Tracking EIS-Related Outcomes 
(Paragraphs 160 & 161) 

In its Phase I assessment of the EIS program, the Monitor found SPD to be in compliance 
with all EIS requirements in the Consent Decree, including data collection and tracking. 
See Monitor's Eighth Systemic Assessment, Dkt. 374 (March 23, 2017). SPD's data 
collection processes have continued in the same manner since that time. Below the 
specific mechanisms employed to collect and track ElS-related data are described. 

• Has the number of individual officers who have triggered EIS reviews 
been completely and accurately collected, maintained, and retrieved 
at the precinct level? 

• Has the number of supervisor reviews of officers based on EIS triggers 
been completely and accurately collected, maintained, and retrieved? 


7 During the study period, EIS policy, section 3.070-POL-2, provided that "an Early Intervention Assessment 
will be conducted at the aggregate level when an employee has a total of six (6) [triggers] between A - M 
during any six-month window when the employee is not on a mentoring plan." The revised policy which 
took effect on January 1, 2019, increased that number to ten triggers within six months. 
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SPD has maintained compliance with these requirements of paragraphs 160 and 161. 
The required data on individual officers and supervisor reviews is collected and 
maintained by the EIS Coordinator and recorded in laPro. During the study period 317 
EIS Assessments (referred to in the Consent Decree as "supervisor reviews") were 
completed. 

Over the study period, 344 unique officers received at least one alert. Approximately 
75 percent of sworn officers did not receive any alerts. More than three-quarters 
(75.87%) of the officers who received at least one alert received fewer than five alerts 
during the period; two officers received more than 30 alerts. See Figure 1. The total 
number of unique officers who received an alert is greater than the number of 
completed assessments (317), because not all of the alerts that occurred during the 
study period had time to work their way through the system to result in a completed 
Assessment. Ultimately, each of these 344 officers will receive an Assessment and 
some will be both placed on a mentoring plan. 

Figure 1: EIS Alerts Per Officer 
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• Has use of force information been completely and accurately collected, 
maintained, and retrieved at the precinct level? 
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The Department has been collecting, maintaining, and reviewing information relating to 
use of force at the precinct level for many years. The Monitor recognized these practices 
in Phase I of the Consent Decree. See Dkts. 231 & 360. More recently, the Department 
has demonstrated its gathering and tracking of such information through its annual Use 
of Force Outcome Reports, which contain a comprehensive overview of all use of force 
for a given year, see Dkt. 524-1, through its Use of Force Reporting and Investigation Audit 
Report (dkt. 497-1), and through its Use of Force dashboard, a version of which is public¬ 
facing 8 . 


• Has information on closed OPA complaints and their dispositions been 
completely and accurately collected, maintained, and retrieved at the 
precinct level? 


As discussed in the Monitor's Fourth Systemic Assessment (Dkt. 259-1 at 18), OPA began 
using lAPro as its day-to-day case management system in 2014. OPA issues annual, public 
reports on this data and its other activities. OPA reports are available at 
https://www.seattle.gov/opa/opa-reports 

Through the EIS module of lAPro, OPA complaints that lead to EIS alerts are tracked and 
reviewed by the chain of command and the PRC for the purpose of making assessments 
and mentoring plans. Because coaching and mentoring are most effective when they are 
conducted close in time to the underlying incident, the EIS process is not delayed to await 
the outcome of OPA's review. Therefore, an officer's assessment or mentoring plan is 
frequently completed before OPA has finished investigating or reviewing the underlying 
complaints. 

Separate and aside from the EIS program, OPA findings are used in other ways by SPD. 
After a recommended finding of sustained has been issued by the OPA Director on an 
allegation of misconduct, the Chief of Police decides what discipline to impose on the 
named employee. If the Chief decides not to follow one or more of the OPA Director's 
recommended findings, the Chief must provide a written statement of the reasons for 
the decision within 30 days of making that decision. 

OPA training recommendations are always followed by SPD. Sometimes the disciplinary 
action recommendation includes a training requirement, which SPD requires the officer 
to complete regardless of the ultimate disciplinary resolution. In addition, for not 


8 https://www.seattle.gov/police/information-and-data/use-of-force-data/use-of-force-dashboard) 
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sustained complaints, OPA may recommend that an officer undergo additional training. 
Officers must complete the recommended training in that situation as well. 

• Has information related to supervisor, precinct, squad and unit trends 
with respect to EIS been completely and accurately collected, 
maintained, and retrieved? 


In the following section, the Department reports on data relating to EIS tracking and alerts 
as required under Paragraphs 160 and 161 of the Consent Decree. 

During the study period (January 1, 2017 to June 30, 2018), a total of 1,316 EIS Alerts 
("alerts") were generated. A breakdown of these alerts, by trigger, is presented in Figure 
2 . 


Figure 2: EIS Alerts by Trigger 9 



Over the study period, 23% of alerts led to an Assessment. That is because many of the 
alerts are combined into one assessment, since the EIS system does not reset after an 
officer has alerted (as explained above in footnote 5). In addition, officers can receive 


9 Aggregate alerts are represented on Figure 2 based on the final trigger that generated the alert. That 
is because EIS "aggregate" alerts are based on multiple different types of triggers. During the study 
period, six different triggers of any type within six months would generate an "aggregate" alert. As of 
January 1, 2019, that number has been increased to ten within six months. 
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multiple alerts for the same underlying incident. To illustrate, assume an officer uses 
Type I force. That single use of force causes the officer to receive three alerts: one alert 
because it is the officer's sixth 10 Type I use of force in six months; one alert because it is 
the officer's sixth trigger (of any type) in six months; and one alert because the subject 
of force made an OPA complaint. In this instance, the officer's supervisor is assigned to 
complete one Assessment—not three Assessments. Finally, some alerts are exact 
duplicates of prior alerts (largely due to how the EIS module handles reclassifications of 
incidents.) The EIS Coordinator reviews each and every new alert and the underlying 
triggers and ensures that multiple EIS Assessments or mentoring plans are not 
completed for the same officer based on the same underlying triggers. 


The two incident types that caused the largest portion of alerts are "Level 1 - Use of 
Force" and "OPA - Investigation." "Level 1 - Use of Force" refers to Type I (least serious) 
force. During the study period, Type I force included, for example, the pointing of a 
firearm, discomfort caused by handcuffing, and other force that causes transitory pain or 
a complaint of transitory pain. * 11 Seattle Police Manual § 8.050. As shown in Figure 2, Type 
I uses of force caused 455 EIS alerts. 

"OPA - Investigation" refers to complaints received by OPA that have been classified for 
investigation (i.e., OPA has determined that at least an initial investigation is warranted). 
OPA complaints classified for investigation caused 375 EIS alerts. 

The vast majority (86.3%) of alerts were generated by officers assigned to the Operations 
Bureau, with West Precinct officers represented most frequently (31.33%), followed by 
North Precinct officers (26.37%). See Table 1. 


10 This is the current threshold for Type I force. For most of the study period the threshold was lower. 

11 As explained above, minor discomfort caused by properly applied handcuffs is no longer classified as 
Type I force. 
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Table 1: Distribution of EIS Alerts by Organizational Assignment 


Officer Bureau Desc Officer Precinct Desc 

% of Total EIS Alert 

Count 

OPERATIONS BUREAU WEST PCT 

NORTH PCT 

SOUTH PCT 

EAST PCT 

SOUTHWEST PCT 

Null 

31.33% 423 

26.37% 356 

13.56% 183 

10.37% 140 

3.56% 48 

1.11% 15 

Total 

86.30% 1165 

HOMELAND SECURITY METROPOLITAN SECTION 

AND SPECIAL TRAFFIC SECTION 

OPERATIONS BUREAU HOMELAND SECURITY SECTION 

3.19% 43 

3.11% 42 

0.44% 6 

Total 

6.74% 91 

Null Null 

HUMAN RESOURCES SECTION 

COMMUNICATIONS SECTION 

1.70% 23 

2.52% 34 

0.15% 2 

Total 

4.37% 59 

PROFESSIONAL TRAINING AND EDUCATION SECTION 

STANDARDS BUREAU FORCE INVESTIGATIONS SECTION 

1.26% 17 

0.07% 1 

Total 

1.33% 18 

INVESTIGATIONS SPECIAL VICTIM CRIMES SECTION 

BUREAU COORDINATED CRIMINAL INVESTIGATIONS SECTION 

VIOLENT CRIMES SECTION 

INTELLIGENCE AND TACTICAL ANALYSIS SECTION 

NARCOTICS SECTION 

0.37% 5 

0.30% 4 

0.30% 4 

0.07% 1 

0.07% 1 

Total 

1.11% 15 

ADMINISTRATIVE Null 

0.07% 1 

SERVICES Total 

0.07% 

BUDGET AND FINANCE Null 

0.07% 1 

Total 

0.07% ~~il 

Total 

100.00% 1350 


The distribution of alerts by organization assignment closely mirrors calls for service, 
crime, force, Terry stops and crisis activity, suggesting no obvious pattern. In other 
words, the highest volume of EIS alerts occurred in West Precinct and North Precinct, 
which are the precincts with by far the highest volume of 911 calls, crime rates, and thus 
policy activity. 

By rank/position of employment, Police Officers accounted for nearly 80% (79.41%) of all 
alerts generated during the study period. Twenty (1.48%) alerts were generated for 
civilian employees, 18 of which were generated for Parking Enforcement Officers. See 
Table 2. 
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Table 2: Distribution of EIS Alerts by Rank/Employee Position 



% of EIS Alert 
Total Count 

Sworn POLICE OFFICER 

POLICE OFFICER PROBATION 

POLICE SERGEANT 

POLICE STUDENT OFFICER 

ASSISTANT CHIEF OF POLICE 

POLICE OFFICER DETECTIVE 

ACTING POLICE SERGEANT 

POLICE LIEUTENANT 

ACTING POLICE OFFICER DETECTIVE 
POLICE - RESERVE OFFICER 

POLICE CAPTAIN 

POLICE COMM DISPATCHER I 

POLICE SERGEANT DETECTIVE 

79.41% 1072 

8.59% 116 

5.19% 70 

1.26% 17 

0.67% 9 

0.59% 8 

0.44% 6 

0.37% 5 

0.15% 2 

0.15% 2 

0.15% 2 

0.15% 2 

0.15% 2 

Total 

97.26% 1313 

Civilian PARKING ENFOR OFFICER 

ACCOUNTING TECHNICIAN II 

ADMIN STAFF ANALYST 

1.33% 18 

0.07% 1 

0.07% 1 

Total 

1.48% 20 

Null Null 

1.26% 17 

Total 

1.26% 17 

Total 

100.00% 1350 


Descriptive analysis suggests no obvious pattern at the Department-wide level, for EIS 
alert generation. Distribution of alerts by assignment, across age and experience closely 
mirror the demographics of the Operational Bureau, suggesting the most frequently 
observed alert activity is the predictable result of police service (complaints, force, vehicle 
operations). 

C. Objective 3 - Assess How EIS Is Being Implemented (Paragraphs 162 
& 163) 

• Are supervisors periodically and appropriately reviewing EIS activity of officers 
in their chain of command? 

Aside from mentoring plans, EIS Assessments are the primary way that supervisors review 
the EIS activity of their employees. Sergeants are also required to monitor their 
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employees' EIS activity on electronic dashboards weekly, and in this way they learn when 
an officer is approaching EIS thresholds. However, Assessments make up the vast majority 
of all work produced by the EIS program. Each Assessment requires an in-person meeting 
between the employee and their supervisor in which the supervisor reviews each of the 
triggering events with the employee. This type of supervisor-employee engagement 
provides the opportunity for a two-way conversation about the employee's work 
performance, well-being, and more. Supervisor-employee engagement is an important 
component for supervisors to guide, counsel, and mentor their employees. An 
Assessment may or may not result in a mentoring plan however the supervisor-employee 
engagement is central to employee management. 

Once the Assessment is completed by the employee's supervisor, it is routed through the 
chain of command. The chain of command - from the supervisor through the 
commanders - must review it and make a determination on the following basic question. 
Do the events indicate that an improvement in performance could be achieved with a 
mentoring plan? 

The majority of Assessments come back with the "No action box checked," meaning that 
no formal mentoring plan is required, which is expected in a threshold-based system. 
However, checking the no action box does not mean that nothing was done to address 
issues and or coach and mentor the employee. In fact, a tremendous amount of good 
leadership can be found documented on these forms. To conduct a proper and accurate 
assessment, a supervisor must review the cases, speak with the employee about the 
triggers, discuss policy, document employee understanding and provide justification for 
decisions made. If the supervisor and chain of command does not properly perform and 
document an Assessment, the PRC (the executive committee which reviews and approves 
all assessments) will return the Assessment for further work. This system fosters careful 
attentive leadership skills in our supervisors. 

All of these Assessments are reviewed through the chain of command and all must meet 
PRC approval. PRC review yields a fairly consistent return rate of about 10 percent for 
Assessments that are not approved. Typical reasons for Assessment returns are for minor 
corrections on format, or inadequate review of triggers. In addition, the chain of 
command and the PRC review the supervisor's determination as to whether or not a 
mentoring plan is warranted. If the supervisor determines there should not be a 
mentoring plan and the chain of command orthe PRC disagrees, then the supervisor must 
write a mentoring plan. The supervisor must then return the Assessment or mentoring 
plan within a specified time period. It will then be reviewed again by the chain of 
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command and the PRC and only proceed if the requested corrections were made in a 
satisfactory way. 

Out of a random sample of 15 EIS cases that involved mentoring plans, in 6 of the cases 
the chain of command determined that no mentoring plan was warranted, but the PRC 
required that a mentoring plan be completed. In one case, the EIS Coordinator returned 
a mentoring plan for the sergeant to make substantive improvements. In two cases, the 
chain of command returned one or more documents for the sergeant to make substantive 
improvements. 

Throughout the study period, the EIS Coordinator has observed a steady increase in 
acceptance and recognition of the goals and worthwhileness of mentoring plans by the 
chain of command, demonstrating a positive development in supervision and leadership. 


• Do supervisors review the progress of assigned intervention strategies as 
appropriate? 

• Are EIS intervention strategies being implemented in a timely manner? 


SPD policy and the practices of the PRC ensure that EIS intervention strategies are 
implemented in a timely manner. Once an EIS Assessment is assigned to a supervisor, the 
supervisor is required to complete the Assessment within fourteen days; the lieutenant 
and captain have 3 and 7 days respectively for their review. If a mentoring plan is 
warranted, then the supervisor is assigned to complete it within a specified amount of 
time (ranging from 10-24 days depending on complexity). Each mentoring plan contains 
a timeline within which the supervisor and chain of command must complete the 
intervention strategy; the most common mentoring plan timeline is 30 days and the 
longest is 90 days. The EIS coordinator reviews and monitors all in-progress mentoring 
plans regularly to ensure that they are closed out in a timely manner. 

Out of a random sample of 15 mentoring plans from the study period (each for a different 
officer), the tasks specified in eight of them had been completed by the officer and 
supervisor within four months. Three additional mentoring plans took between four and 
five months to complete. One mentoring plan, created to address OPA complaints 
involving allegations of bias and excessive force, took 181 days to complete. These 
timeframes were all appropriate, given the subject matter being addressed. For example, 
a mentoring plan to address routine driving issues can be completed relatively quickly, as 
it typically involves taking a driving course that is offered regularly. Interventions assigned 
to address more complex or weighty topics appropriately take longer and involve 
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substantially more interactions over a longer period of time between the officer and his 
or her supervisor. 

Although three of the fifteen plans were not completed, there were legitimate reasons 
for each. In one, the officer went on an extended leave and the mentoring plan was placed 
on hold. For the other two, the officers did not complete the interventions and the EIS 
cases were ultimately closed out because the chain of command determined that the 
disciplinary process would be the more appropriate approach to correct the identified 
issue. 

For the 317 assessments that were assigned and completed during the study period, more 
than one third were completed with one week of the initial EIS alert. All but 14 were 
completed within two months. 

Table 2: Distribution of EIS Alerts by Rank/Employee 


El Assessments - Days to Complete 



As with the underlying Assessments, mentoring plans provide an opportunity for 
continued supervisor-employee engagement. The mentoring plan process requires that 
the supervisor meet in person with the officer and, where appropriate, the supervisor 
must participate in the assigned intervention. For example, the recommended action step 
for an Assessment related to receipt of an OPA Complaint is "[r]ide along with employee 
for a minimum duration of one hour per shift, twice per work week." The EIS Assessment- 
Mentoring Plan Form contains other suggested intervention strategies and is attached to 
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this report as Appendix C. However, there is no formula that a supervisor can apply when 
determining an appropriate intervention; the decision must be based on the supervisor's 
interactions with and knowledge of the welfare and status of their employees. In addition, 
the EIS Coordinator maintains a large repository of high-quality Assessments and 
mentoring plans that have been anonymized to provide instructive examples. 

• Is data regarding interventions being tracked in EIS? 

For every EIS Assessment or mentoring plan, a "final disposition form" is entered into 
lAPro. Using this form, the Department records the specific interventions assigned, if any, 
and the dates on which the assessments or interventions are completed. 

In sum, the Department has continued to sustain compliance with each of Paragraphs 157 
through 163 of the Consent Decree. 

FUTURE STEPS 

The findings of this audit reveal significant challenges and critical areas for improvement 
in the EIS program. More broadly, as the social science around early intervention theory 
has evolved substantially over the past several years, SPD must ensure that its system 
evolves to keep pace. Committed to doing so, the Department describes below the work 
that it is doing in this regard, independently and in collaboration with partners, nationally. 

In the near-term, the Department's next EIS Audit, scheduled to be completed in 
December 2019, will analyze the effects on the EIS program of the changes to the 
definition of Type I force and adjustments to EIS triggers and thresholds. 

The Department is also taking a leadership role nationally by exploring ways to leverage 
multi-disciplinary social science research to improve upon the EIS model. In concept, Early 
Intervention Systems are designed to leverage data-based metrics to identify officers at 
risk of future performance issues, providing police supervisors an opportunity to 
intervene through mentoring, counseling, additional training, or other non-disciplinary 
measures at a point before the officer is involved in a significant adverse event. Such 
systems are intended to promote chain of command accountability for employee 
performance while also providing supervisors a foundation to foster and recognize the 
success and professional growth of their employees. 

In recent years, the model of tying EIS review to the occurrence of policing activities, such 
as the use of force, has been criticized in academic literature as ineffective, often 
counterproductive, and perceptually, as punitive and stigmatizing. See, e.g., Worden et 
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al. (2013) Intervention with Problem Officers: An Outcome Evaluation of an EIS 
Intervention. Crim. Justice and Behavior, Vol. 40, No. 4 409-437; 
http://www.latimes.com/local/la-me-lapd-problem-officers-20140826-story.html. 

SPD fully endorses the importance of developing an effective early intervention model, 
from both an officer safety/well-being and Department risk management perspective. 
Accordingly, while appreciating that designing an accurate model for attempting to 
predict how any one individual will behave in dynamic, unpredictable, and often 
dangerous situations is an inherently complex and iterative task, SPD is actively engaged 
in efforts on multiple fronts to further advance the social science underlying this goal. At 
the national level, SPD is a core contributing member of a multi-disciplinary advisory 
committee, comprising representatives of law enforcement, medical and mental health 
fields, and research institutions from around the country, working in collaboration with 
the Chicago Police Department, the Los Angeles Police Department, and the University of 
Chicago Crime Lab to design a next-generation EIS. At the organization level, continuing 
its work with research partners from Washington State University, SPD is exploring how 
it might best leverage the increasing sophistication of its data and analytic tools to allow 
for more validated identification of relevant variables to an EIS, both psychological and 
situational. 

Paramount in both efforts is a shift of focus away from the outcome of an incident 
(outcomes that, again, are subject through separate processes for timely and rigorous 
critical review) and instead towards the nature of incidents to which officers are called to 
respond. The premise for this shift is grounded in the impacts - physiologically, 
psychologically, and behaviorally - on cumulative incident stress and vicarious trauma on 
first responders. In simplistic form, the Department's longer-term vision for EIS is built 
on the following premises: 

1. By nature of their occupation, law enforcement officers are regularly exposed to 
incident stress likely to manifest, acutely or over time, in secondary traumatic 
stress/vicarious traumatization. 12 


12 "Vicarious trauma" has been explained as 

[A]n occupational challenge for people working and volunteering in the fields of victim services, 
law enforcement, emergency medical services, fire services, and other allied professions, due to 
their continuous exposure to victims of trauma and violence. This work-related trauma exposure 
can occur from such experiences as listening to individual clients recount their victimization; 
looking at videos of exploited children; reviewing case files; hearing about or responding to the 
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2. Repeated stressful and/or traumatic events can chronically elevate the body's stress 
response, leading to poorer physiological and psychological functioning, up to and 
including post-traumatic stress disorder (PTSD). 13 

3. Recognizing, mitigating, and ideally preventing vicarious trauma and its sequelae is 
both a critical officer safety issue 14 and an organizational risk management priority. 15 

4. If the goal of an EIS is to identify potentially problematic performance indicators 
before they culminate in an adverse (out of policy, or worse) event, then such a system 
must be: 


aftermath of violence and other traumatic events day after day; and responding to mass violence 
incidents that have resulted in numerous injuries and deaths. 

Vicarious trauma overlaps with concepts of secondary traumatic stress (the "natural consequent 
behaviors and emotions that often result from knowing about a traumatizing event experienced by 
another and the stress resulting from helping, or wanting to help, a traumatized or suffering person[,] 
with symptoms [that] can mimic those of posttraumatic stress disorder") and burnout ("a state of 
physical, emotional, and mental exhaustion caused by long-term involvement in emotionally 
demanding situations."). 

https://vtt.ovc.oip.gov/what-is-vicarious-trauma 

13 Such symptoms include difficulty managing emotions, difficulty making good decision, problems 
managing boundaries, sleep disruption and/or nightmares, generalized hypervigilance/anxiety, substance 
abuse and engaging in high risk behaviors, changes in appetite, negativity, and social isolation. Some 
studies estimate that up to nearly 20% of police officers in the United States meet DSM-V diagnostic 
criteria for PTSD; other studies estimate that up to 34% suffer symptoms associated with PTSD, although 
they may not meet criteria for full diagnosis. See, e.g., Charles R. Marmar, et al., Predictors of 
Posttraumatic Stress in Police and Other First Responders, 1071 ANNALS N.Y. ACAD. SCI. 1, 18 (2006); 
Regehr., C. and LeBlanc, VR (2017, PTSD, Acute Stress, Performance and Decision-Making in Emergency 
Service Workers. J. Am. Acad. Psychiatry and Law. 45(2) 184-192. 

14 See, e.g., President's Task Force on 21 st Century Policing, Washington DC Office of Community Oriented 
Policing Services, Final Report of the President's Task for on 21 st Century Policing (2015) 
http://www.cops.usdoi.gov/pdf/taskforce/taskforce finalreport.pdf (noting the toll of stress on the life 
expectancy of law enforcement officers). 

15 See generally Regehr and LeBlanc (2017), discussing the impact of stress conditions on perception, 
cognition, risk assessment, and response. 
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a. tied to indicators that are at minimum logically, but where possible, empirically, 
predictive of future performance deficits; and 

b. structured so as to provide meaningful mitigation strategies that address both the 
nature of the indicators and, to the extent that the Department may influence, 
physiological/psychological factors that may negatively impact an officer's 
performance, professional growth, development, and well-being. 

This vision is not only confidently aligned with core principles around preserving 
performance ideals in first responders, but importantly, incorporates feedback received 
from officers, supervisors, and commanders around advancing an EIS that truly succeeds 
in identifying and mitigating against predictors of future problematic performance. 
Indeed, there were positive points elicited from focus groups that must continue to be 
nurtured in future iterations of EIS. As one example, supervisors expressed a general 
hunger for aggregated information around their employees' overall activity and a 
mechanism for identifying trends within their squads. Officers expressed appreciation for 
a system that encouraged conversation with their supervisors; supervisors likewise 
expressed a desire for greater interaction with their officers but lamented the limited 
time available given the amount of paperwork that they are also responsible to complete. 
And importantly, officers across the ranks of focus groups - but particularly those in 
senior positions-emphasized the significance, in time and overtime, of vicarious trauma, 
critical incident stress, and the toll of this experience on officer well-being and 
performance, and openly advocated for a system that would shift an EIS in this direction. 

Over the longer term, SPD intends to continue its engagement with organizations 
nationally, with research partners, with its employees, and with the Inspector General to 
develop a new model for EIS that more accurately meets the goals of the system. 

VALIDATION 

In Phase I of the work of under the Consent Decree, DOJ and the Monitoring Team 
reviewed SPD's compliance with the requirements of the Consent Decree through 10 
assessments, covering the roughly six topic areas of the Consent Decree: force 
investigation and reporting, crisis intervention, supervision, Early Intervention System 
("EIS"), use of force, and stops and biased policing. By the end of 2017, the Monitoring 
Team and DOJ found the City of Seattle to be in compliance with each area. On that basis, 
the Court issued a finding of "full and effective compliance" with the requirements of the 
Consent Decree. By the terms of the Consent Decree, the City of Seattle is now required 
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to demonstrate that it can sustain compliance with those requirements for a period of 
two years. 

During Phase II of the Consent Decree work, the City of Seattle has taken over the lead 
role in conducting assessments of the six core topic areas of the Consent Decree. By taking 
this lead role, SPD must demonstrate not only sustained compliance, but also a 
willingness and ability to critically self-assess their own progress in these areas, which are 
central to effective and constitutional policing. 

This does not mean, however, that the work of DOJ and the Monitoring Team is done. In 
Phase II, DOJ and the Monitoring Team are reviewing the City's proposed methodologies 
for each audit and are conducting their own independent research and analysis or "look 
behind" the City's review. 

Forthis audit, DOJ and the Monitoring Team consulted with SPD and ultimately approved 
the methodology used by SPD in conducting its own self-assessment. Further, as part of 
their own validation review, DOJ and the Monitoring Team interviewed the EIS 
Coordinator regarding (1) the process and decisionmaking involved in converting EIS 
triggers to assessment assignments for supervisors; (2) oversight of supervisory decisions 
to create or not create PMPs from assessments; and (3) oversight over supervisory 
decisions regarding the content of PMPs. DOJ and the Monitoring Team also participated 
in discussions with SPD at the end of 2018 regarding revisions to the EIS triggers and, 
ultimately, approved policy changes related to those triggers in order to support SPD's 
efforts to make the system more usable for risk management purposes. 

In addition, DOJ and the Monitoring Team requested and received a randomly selected 
sample of EIS case files, including the initial assessments, Performance Mentoring Plans, 
Status Reports, and Performance Review Committee Disposition Forms, for the time 
period January 1, 2017 to June 30, 2018. The sample set was comprised of 15 of the 33 
Performance Mentoring Plans reviewed by SPD's Performance Review Committee during 
the study period. DOJ and the Monitoring Team, together with their subject matter 
experts, reviewed these materials for compliance with the terms of the Consent Decree 
relating to EIS and SPD's policies regarding the same. DOJ and the Monitoring Team 
subsequently conferred about their findings and, based on their Phase II reviews, 
concluded as follows: 

• The City of Seattle has demonstrated that it has sustained compliance with the EIS 
requirements of the Consent Decree and SPD's policies governing the same, 
including requirements that the EIS system continue to be used for risk 
management (and not disciplinary) purposes and that supervisors periodically 
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review EIS activity of officers in their chain of command. Further, the City has 
demonstrated (in particular, recently) that it has monitored and made adjustments 
to the EIS system to ensure that it is meeting its risk management objectives. 

• In addition, DOJ and the Monitoring Team commend the City for seeking out and 
collaborating with those in the field of EIS to stay abreast of best practices and 
developments in how better to anticipate and address issues with officers in their 
command prior to their development into major events. DOJ and the Monitoring 
Team will continue to follow these developments as well and will include them in 
conversations about the future of EIS during the next audit period, which is due to 
result in a December 2019 filing with this Court. 

In the interest of continuous improvement, DOJ and the Monitoring Team offer the 
following technical assistance to SPD based upon issues spotted during their respective 
reviews. Although none of these issues rose to the level of systemic non-compliance with 
the terms of the Consent Decree, DOJ and the Monitoring Team strongly encourage SPD, 
and the Inspector General who will take over the audit function for this topic area in the 
future, to give attention to these matters going forward: 

• From our sampling of PMPs, there appeared to be at least a few occasions where 
supervisors were not clear on the amount of discussion they could engage in with 
respect to officers who "triggered" under the EIS system because of (or in part 
because of) complaints that were made about them to the Office of Police 
Accountability ("OPA"). While we understand that collective bargaining rules 
regarding discipline may limit the amount of discussion supervisors can have 
regarding a specific incident while it is being investigated by OPA, supervisors 
should be made aware that they are able to, and should, still thoroughly discuss 
the underlying issues of the complaint. 

• DOJ and the Monitoring Team were particularly impressed with the thoughtful and 
robust manner by which the current EIS Coordinator approaches his role with 
respect to identifying and assigning out EIS "assessments" and evaluating the work 
of supervisors in selecting and crafting PMPs. However, we believe that in order 
for the system to continue to function effectively and with the benefit of lessons 
learned from the EIS Coordinator, there should be more robust systems in place to 
institutionalize that role. To that end, we would encourage SPD to adopt a practice 
of semi-yearly meetings or trainings in which the EIS Coordinator provides a 
detailed account of how his or her work is done to at least 2-3 personnel who 
would be in the position to step into the role of the EIS Coordinator as needed. 
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• During our review of the EIS Status Reports and the Performance Review 
Committee Disposition Forms it was not always clear what actions had been taken 
in response to the PMP and when those actions were completed. While the PRC 
has the ability to communicate directly on these points, including through Blue 
Team, it may be difficult for reviewers/auditors to easily view what has occurred. 
Accordingly, we would recommend issuing clarifying instruction to supervisors and 
the PRC regarding clear documentation of the completion of PMP action items 
using the Status Report and Performance Review Committee Disposition Forms, 
including dates completed. 

• Finally, while we acknowledge the importance of the work the City and those at 
SPD have done to fine tune the EIS systems and its triggers to better serve the City's 
risk management needs, we believe that there is an additional approach that has 
not yet been explored. SPD's findings related to the "over-triggering" of proactive 
officers is consistent with what other police departments have found in the early 
years of their own EIS systems. One well-established fix to this issue is to embed a 
"peer to peer" analysis into the triggers. In other words, instead of setting up a 
trigger for numbers of uses of force across all officers, triggers would be calibrated 
to compare only officers with like assignments (e.g. by a particular watch/precinct, 
or a particular assignment like the West Bike Squad). This approach is much better 
suited to truly identifying outliers. Accordingly, we strongly encourage SPD to 
analyze this issue and to commit it to practice through its next EIS policy review. 
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